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changing words, rules and transitions in a grammar in the 
middleware layer based on the word change data, rule 
change data and transition change data received. 

53. (original) A method of formatting data for use by a speech 
. engine and an audio device, comprising 

obtaining, at a middleware layer which facilitates 
communication between the speech engine and an 
application, a data format for data used by the engine; 

obtaining, at the middleware layer, a data format of data 
used by the audio device; 

determining, at the middleware layer, whether the engine data 
format and the audio data format are consistent; and 

if not, utilizing the middleware layer to attempt to change 
the data format of the data used by at least one of the 
engine and the audio device. 

54. (original) The method of claim 53 and further comprising: 

if the attempt to change the data format used by the at least 
one of the engine and the audio device is unsuccessful, 
invoking a format converter to change data format for 
data between the engine and the audio device to ensure 
the data formats are consistent. 

REMARKS 

This is in response to the Office Action mailed on 
April 28, 2005. Claims 1-39, 47 and 50-54 were pending in that 
action. Claims 47 and 50-54 were indicated as allowable. Claims 
1-3 and 6-39 were rejected. Claims 4 and 5 were objected to. With 
this response, all claims remain unchanged but for a minor 
amendment to the dependency of claim 50. 
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In this case, on October 1, 2004, Applicant received a 
first Office Action wherein the only substantive rejection based 
on cited prior art was against claims 40-48. On pages 8-11 of 
that action, the Examiner provided Applicant with instructions as 
how minor adjustments could be made to place claims 1-39 and 49- 
54 in condition for allowance. In reliance on this indication of 
allowable subject matter, Applicant prepared and filed a response 
on November 23, 2004. In this response, Applicant canceled all 
substantively rejected claims and, for the remaining claims, 
addressed all noted hindrances to allowability. In ; '.addition, it 
is at least worth noting that Applicant also filed additional 
patent applications that incorporated assumptions based on the 
Examiner ' s comments in the Office Action of October 1, 2004. 

Under the circumstances, Applicant was surprised to 
receive the latest Office Action of April 28, 2005, which 
essentially revokes the previous indication that claims 1-39 are 
in allowable form, and establishes a substantive rejection 
against those claims. Section 706.04 of the Manual of Patent 
Examining Procedure (M.P.E.P.) mandates that great care should be 
exercised in submitting a rejection of previously allowed claims. 

Applicant certainly respects the fact that the Examiner is 
entitled to a change of mind, especially when new or additional 
art is encountered. However, notably, every reference cited 
against the claims in the latest Office Action was noted without 
rejection in the previous Office Action of October 1, 2004. 
Applicant is at a loss to understand why the rejections based on 
these references were not previously established. 

Nevertheless, for reasons that will be explained in 
detail below, it is respectfully submitted that the cited 
references, considered independently or in combination, fail to 
teach or suggest the elements of the rejected claims 1-3 and 6- 
39. This being the case, Applicant will request that the 
rejections be withdrawn and that this case be placed on track for 
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allowance, consistent with Applicant's expectations following the 
previous Office Action. Applicant respectfully requests 

reconsideration in light of the comments that follow. 

Applicant hereby affirms the indication in the latest 
Office Action that claims 47 and 50-54 are in allowable form. 
With the present response, a minor amendment has been made to 
claim 50 to correct what Applicant believes is a minor error to 
the stated dependency. Consideration and entry of this amendment- 
are respectfully solicited. Applicant also affirms the indication 
that claims 4 and *5 would be allowable if re-written. ;Atv.*this 
time, Applicant has not re-written those claims based on a 
belief, to be described below, that the base and intervening 
claims also contain allowable subject matter. It is noted that 
claims 40-46, 48 and 49 were previously canceled. 

Thus, the remaining issues in the prosecution of the 
present application pertain to claims 1-3 and 6-39. Of these 
claims, only claim 1 is independent. Claims 2, 3 and 6-39 are 
dependent upon claim 1. 

On page 3 of the Office Action, independent claim 1 is 
rejected under 35 U.S.C. §102 (a) as being anticipated by the 
Hataoka reference. For reasons that will now be discussed in 
detail, it is respectfully submitted that the Hataoka reference 
fails to teach or suggest the elements of claim 1. 

Claim 1 recites a middleware layer configured to 
facilitate communication between a speech-related application and 
a speech-related engine. As claimed, the middleware layer 
includes a speech component having an application- independent 
interface and an engine -independent interface. The claimed 
application- independent interface is configured to be coupled to 
the application. The claimed engine -independent interface is 
configured to be coupled to the engine. 

While the Hataoka reference does use the term 
"middleware" to describe a collection of speech processing 
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components, the described middleware is fundamentally different 
than the claimed middleware. The middleware described in the 
Hataoka reference specifically includes a speech recognition 
module and a speech synthesis module (see abstract) . As is 
clearly shown in Table 1 on page 693 of the cited reference, the 
described middleware specifically includes recognition and 
synthesis engines. As a complete reading of the Hataoka 
reference makes clear, the inclusion of the engines in the 
middleware layer makes sense because the described purpose of the 
middleware software is to provide a layer of speech-oriented 
support that serves as a connection between related hardware and 
user applications (see introduction) . It is specifically stated 
in the introduction of the Hataoka reference that the processing 
middleware consists of speech recognition and speech synthesis 
software on a microprocessor. 

There are multiple reasons why the elements of claim 1 
are distinguishable from the cited reference. An obvious 
difference is that the cited reference in no way teaches or 
suggests an engine -independent interface. As has been 

established, the "middleware" described in the Hataoka reference 
includes specific recognition and synthesis components. The 
claim recites a middleware layer that, as claimed, is essentially 
independent from any requirement of a specific speech-related 
engine. The cited reference neither teaches nor suggests any 
such separation of components. 

Applicant believes that it is possible that the 
Examiner may be misunderstanding the function of the middleware 
described in the Hataoka reference. For example, in paragraph 13 
of the latest Office Action, the Examiner describes the Hataoka 
"middleware" as acting as an intermediary between applications 
and engines. This is a mischaracterization of the Hataoka 
middleware. The Hataoka middleware includes the engines. In 
fact, as is clearly stated in the introduction and in multiple 
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places throughout the Hataoka reference, the Hataoka middleware 
acts as an intermediary to connect hardware (assumedly including 
speech-oriented input and output mechanisms) to user 
applications. In essence, the Hataoka middleware is a 
microprocessor equipped with software that enables the 
microprocessor to deliver speech functionality that operates 
between a user application (such as car navigation systems, 
handheld PCs, game machines, etc.) and corresponding hardware. 
In no way is the Hataoka middleware software engine- independent . 
In fact, it is engine -dependent . ... . . 

In order to reject a claim under 35 U.S.C. §102, the 
examiner must find every limitation of the claim in the cited 
reference. As has been discussed, Applicant submits that 
independent claim 1 recites limitations that are neither taught 
nor suggested by the Hataoka reference. Thus, it is respectfully 
submitted that claim 1 is allowable in its present form. 

Further, it is respectfully submitted that dependent 
claims 2, 3 and 6-39 are in allowable form at least based on 
their dependence upon what Applicant believes to be an allowable 
claim. Further, it is respectfully submitted that many of 
dependent claims 2, 3 and 6-39 recite additional elements that 
qualify the claims to be deemed independently allowable. To the 
extent that the Examiner relies on other references (i.e., the 
Baker and Comerford references) under 35 U.S.C. §103, it is 
respectfully submitted that those references do not remedy the 
inability of the Hataoka reference to teach or suggest the 
elements of the claims. 

Still further, according to Section 2142 of the 
M.P.E.P., the examiner bears the initial burden of factually 
supporting any prima facie conclusions of obviousness. In order 
to establish a prima facie case of obviousness in instances where 
multiple references are cited in combination, the Examiner must 
show 1) that a suggestion exists for combining the references, and 
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2) that the combined references teach or suggest all the recited 
claim limitations. Id. For reasons that will be discussed 
thoroughly below, with regard at least to claims 3, 6-9, 12, 17, 
19, 22, 23 and 25-35, it is respectfully submitted that the 
Examiner in the present case has failed to support any prima facie 
conclusion of obviousness. 

The rejection of each of claims 3, 6-9, 12, 17, 19, 22, 
23 and 25-35 is based on a rationale of obviousness under 35 
U.S.C. §103. The rejection is based one of the Baker and 
Comerford references in combination with the Hataoka reference. 
In support of rejection, the only stated motivation for combining 
references is a simple statement of benefit associated with one of 
the references in the combination (i.e., "because it would allow 
the system to use a grammar associated with the application hence 
giving better results" or "because it would enable the system to 
evaluate the surrounding vocabulary hence allowing the best 
recognition result to be obtained") . 

It is fundamental that rejections under 35 U.S.C. §103 
must be based on evidence comprehended by the language of that 
section. In re Grasselli , 713 F.2d 731, 739, 218 USPQ 769, 775 
(Fed. Cir. 1983) . The factual inquiry as to whether an invention 
is obvious in view of prior art must be based on objective 
evidence of record. In re Lee , 61 USPQ2d 1430, 1433 (Fed. Cir. 
2002) . 

In the present case, the Examiner has provided no 
reasonable rationale as to why it would be obvious to combine the 
cited references. In fact, the three cited references are related 
to three applications within the art that are remotely related. 
Outside of a single benefit associated with an invention extracted 
from the prior art, the Examiner does not give any indication as 
to why one skilled in the art would have been so motivated to 
conceive the proposed combinations. No specific citations are 
made to any of the cited references that would demonstrate a 
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teaching or suggestion to support a finding that the claimed 
features are obvious. No explanation is provided as to what 
knowledge possessed or specific principle known by a skilled 
artisan would lead to conception of the claimed features. No 
recognition is given to applicable trends in the art. No 
explanation is given as to how or why one skilled in the art would 
be lead to the claimed invention. 

Applicant has been left to assume that the only 
motivation for the alleged obviousness is impermissible hindsight 
gleaned from the present invention. See, e.g., Ex parte Haymond , 
41 USPQ2d 1217, 1220 (BdPatApp&Int 1996) (the examiner may not, 
because he doubts that the invention is patentable, resort to 
speculation, unfounded assumptions or hindsight reconstruction to 
supply deficiencies in the factual basis) . Simply stated, the 
Office Action of record does not provide any objective evidence 
that shows a motivation as to why Applicant's claims 3, 6-9, 12, 
17, 19, 22, 23 and 25-35 are obvious in view of the combination of 
cited references. Applicant therefore submits that no prima facie 
case of obviousness has been established. Applicant respectfully 
submits that these claims are allowable for this additional 
reason. 

Still further, in the latest Office Action, in the 
rejection of at least claims 9, 14, 21, 24 and 39, claim elements 
missing from the cited prior art are accounted for based on a 
finding of Official Notice. Consistent with M.P.E.P. §2144.03, 
Official Notice unsupported by documentary evidence should only 
be taken by the examiner where the facts asserted to be well- 
known, or to be common knowledge in the art are capable of 
instant and unquestionable demonstration as being well-known. 
Even assuming, arguendo , that the missing elements from 
Applicant's claims are known in the art, Applicant rejects any 
assumption that such elements are known in their claimed context 
of a middleware environment, especially considering the claimed 
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application- and engine -independent interfaces. Consistent with 
§2144.03, Applicant hereby traverses all assertions of Official 
Notice in the latest Office Action and requests that Examiner 
support such findings with adequate evidence. 
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In summary, it is respectfully submitted that all 
pending claims, namely, 1-3 9, 4 7 and 50-54 are in condition for 
allowance. Consideration and allowance of these claims are 
respectfully solicited. The Director is authorized to charge any 
fee deficiency required by this paper or credit any overpayment 
to Deposit Account No. 23-1123. 

Respectfully submitted, 
WESTMAN, CHAMPLIN & KELLY, P. A. 
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